Monitoring, analysing and predicting network performance in grids

Monitoring, Analysing and Predicting Network Performance in Grids by Yehia El-khatib M.Sc., Lancaster University, 2006 B.Sc., Alexandria University, 2002 Thesis submitted for the degree of Doctor of Philosophy September 2011 School of Computing and Communications Lancaster University The grid computing paradigm has facilitated the instrumentation of complex, highly-demanding collaborative applications. The technologies that make grid computing possible have mostly evolved from parallel and cluster systems. Although this has certainly empowered the grid computing field, part of the heritage has been the perception that required network resources are taken for granted. This is precarious, considering that most grids rely on public IP networks, like the Internet, as the underlying network. This assumption has obstructed the path of grid computing. This thesis aims to improve the performance of grid applications by facilitating network-aware grid scheduling. This is achieved by providing network performance information to grid schedulers, allowing them to adapt to changes in the network. The contribution of this thesis is twofold: a novel approach to network measurement that is particularly suitable for grid environments; and a distributed system that collects and manages these measurements, predicts future network performance, and disseminates this information to schedulers. The accuracy and effectiveness of this system is evaluated on a production grid infrastructure used for e-science applications. The outcomes of this evaluation provide a strong argument for the introduction of network-aware grid schedulers, information systems, and job and resource description standards.